The present invention relates to computer graphics, and more particularly to texture methods and storage.
Interactive video games use computer graphics to generate images according to game application programs. FIG. 2a illustrates typical stages in computer graphics rendering which displays a two-dimensional image on a screen from an input application program that defines a virtual three-dimensional scene. In particular, the application program stage includes creation of scene objects in terms of primitives (e.g., small triangles that approximate the surface of a desired object together with attributes such as color and texture); the geometry stage includes manipulation of the mathematical descriptions of the primitives; and the rasterizing stage converts the three-dimensional description into a two-dimensional array of pixels for screen display.
FIG. 2b shows typical functions in the geometry stage of FIG. 2a. Model transforms position and orient models (e.g., sets of primitives, such as a mesh of triangles) in model/object space to create a scene (of objects) in world space. A view transform selects a (virtual camera) viewing point and direction for the modeled scene. Model and view transforms typically are affine transformations of the mathematical descriptions of primitives (e.g., vertex coordinates and attributes) and convert world space to eye space. Lighting provides modifications of primitives to include light reflection from prescribed light sources. Projection (e.g., a perspective transform) maps from eye space to clip space for subsequent clipping to a canonical volume (normalized device coordinates). Screen mapping (viewport transform) scales to x-y coordinates for a display screen plus a z coordinate for depth (pseudo-distance) that determines which (portions of) objects are closest to the viewer and will be made visible on the screen. Rasterizing provides primitive polygon interior fill from vertex information; e.g., interpolation for pixel color, texture map, and so forth.
Programmable hardware can provide very rapid geometry stage and rasterizing stage processing; whereas, the application stage usually runs on a host general purposed processor. Geometry stage hardware may have the capacity to process multiple vertices in parallel and assemble primitives for output to the rasterizing stage; and the rasterizing stage hardware may have the capacity to process multiple primitive triangles in parallel. FIG. 2c illustrates a geometry stage with parallel vertex shaders and a rasterizing stage with parallel pixel shaders. Vertex shaders and pixel shaders are essentially small SIMD (single instruction multiple dispatch) processors running simple programs. Vertex shaders provide the transform and lighting for vertices, and pixel shaders provide texture mapping (color) for pixels. FIGS. 2d-2e illustrate pixel shader architecture.
3D games make use of several hundreds of texture images to try to create realistic 3D environments inside 3D games. These texture images, which take up tens to hundreds of megabytes of storage space, are distributed along with the 3D game. The total storage space used by the 3D game is an important consideration in mobile and portable games since media such as flash memory cards used to distribute the game are relatively expensive and have limited capacity. Game size is also an important consideration for network downloadable games on cellphones and other platforms since bandwidth comes at a premium. Texture images take up a significant portion of the total memory used by a game. Hence efficient ways to compress texture images for game distribution becomes important. Compressing texture images also reduces the system memory requirements of 3D games. On system memory limited systems such as cellphones, texture image compression becomes especially important.
Current approaches use lossy compression schemes such as JPEG or DXTC or wavelet coders to individually compress the texture images before distribution. When quality is important, lossy compression is not used. The texture images may get compressed losslessly by the archival program used to package the game release.
In 3D games, texture images are frequently applied on irregular surfaces such as those of humanoid models, weapons, and other irregular objects. The texture images used in these cases have irregular shapes. Because of system constraints (e.g., regularity of texture memory accesses, et cetera), these irregular shapes are stored in rectangular images as illustrated in FIG. 4a. 
Texture compression is used in 3D graphics to reduce the enormous amount of bandwidth (which can run into several GB/s) required to transfer texture data to the 3D graphics CPU for use in rendering of 3D graphics scenes. The 3D graphics CPU decompresses the texture image during run-time and uses it for rendering. For reasons of computational efficiency and support for random access of texture image data, simple fixed-length compression schemes are used rather than the bit-efficient and complex variable length schemes such as JPEG. Note that JPEG compression is used for texture compression for game distribution. Before a game is run, the JPEG textures are decompressed and the uncompressed data is used in run-time.
DXTC texture compression is the most popular texture compression scheme used in 3D graphics and is supported on virtually all the modern graphics CPUs. DXTC is a fixed-length compression scheme that provides a 1:4 or 1:6 compression ratio (depending on the precision of the input data). In DXTC image compression, 4×4 blocks of input images are encoded using 64 bits. The data encoded for each 4×4 block consists of two colors c0 and c1 in RGB565 format (taking up 32 bits) and 2-bit indices for each of the 16 pixels in the 4×4 block (taking up 32 bits). In the decoder two intermediate colors are produced by linear interpolation of c0 and c1 to give a total of four color (including c0 and c1). The 2-bit index for each pixel decides which of the four colors to use for coloring the pixel. The DXTC compression scheme can be thought of as a vector quantization (VQ) scheme.
When compared to JPEG, DXTC provides a far inferior performance. However, in practice, when there are enough details in the texture image, the quality degradation is not perceived. Instead, there is a perceived improvement in quality since texture compression enables one to use a texture image that is double the original size and this brings more details into the rendered scene. When the texture images are smoothly varying, the distortion is, however, noticeable. If improved quality is desired on current 3D graphics systems, one would have to use uncompressed texture images which require 24-bits for each color.
U.S. Pat. No. 6,768,493 discloses texture storage with compressed sub-blocks of texels in an image plane.
However, these applications have complexity, memory bandwidth, and compression trade-offs in 3D rendering of video clips.